<!-- HTML header for doxygen 1.8.6-->
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=9"/>
<meta name="generator" content="Doxygen 1.8.13"/>
<title>OpenCV: Image Segmentation with Watershed Algorithm</title>
<link href="../../opencv.ico" rel="shortcut icon" type="image/x-icon" />
<link href="../../tabs.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="../../jquery.js"></script>
<script type="text/javascript" src="../../dynsections.js"></script>
<script type="text/javascript" src="../../tutorial-utils.js"></script>
<link href="../../search/search.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="../../search/searchdata.js"></script>
<script type="text/javascript" src="../../search/search.js"></script>
<script type="text/x-mathjax-config">
  MathJax.Hub.Config({
    extensions: ["tex2jax.js", "TeX/AMSmath.js", "TeX/AMSsymbols.js"],
    jax: ["input/TeX","output/HTML-CSS"],
});
//<![CDATA[
MathJax.Hub.Config(
{
  TeX: {
      Macros: {
          matTT: [ "\\[ \\left|\\begin{array}{ccc} #1 & #2 & #3\\\\ #4 & #5 & #6\\\\ #7 & #8 & #9 \\end{array}\\right| \\]", 9],
          fork: ["\\left\\{ \\begin{array}{l l} #1 & \\mbox{#2}\\\\ #3 & \\mbox{#4}\\\\ \\end{array} \\right.", 4],
          forkthree: ["\\left\\{ \\begin{array}{l l} #1 & \\mbox{#2}\\\\ #3 & \\mbox{#4}\\\\ #5 & \\mbox{#6}\\\\ \\end{array} \\right.", 6],
          forkfour: ["\\left\\{ \\begin{array}{l l} #1 & \\mbox{#2}\\\\ #3 & \\mbox{#4}\\\\ #5 & \\mbox{#6}\\\\ #7 & \\mbox{#8}\\\\ \\end{array} \\right.", 8],
          vecthree: ["\\begin{bmatrix} #1\\\\ #2\\\\ #3 \\end{bmatrix}", 3],
          vecthreethree: ["\\begin{bmatrix} #1 & #2 & #3\\\\ #4 & #5 & #6\\\\ #7 & #8 & #9 \\end{bmatrix}", 9],
          cameramatrix: ["#1 = \\begin{bmatrix} f_x & 0 & c_x\\\\ 0 & f_y & c_y\\\\ 0 & 0 & 1 \\end{bmatrix}", 1],
          distcoeffs: ["(k_1, k_2, p_1, p_2[, k_3[, k_4, k_5, k_6 [, s_1, s_2, s_3, s_4[, \\tau_x, \\tau_y]]]]) \\text{ of 4, 5, 8, 12 or 14 elements}"],
          distcoeffsfisheye: ["(k_1, k_2, k_3, k_4)"],
          hdotsfor: ["\\dots", 1],
          mathbbm: ["\\mathbb{#1}", 1],
          bordermatrix: ["\\matrix{#1}", 1]
      }
  }
}
);
//]]>
</script><script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js"></script>
<link href="../../doxygen.css" rel="stylesheet" type="text/css" />
<link href="../../stylesheet.css" rel="stylesheet" type="text/css"/>
</head>
<body>
<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
<div id="titlearea">
<!--#include virtual="/google-search.html"-->
<table cellspacing="0" cellpadding="0">
 <tbody>
 <tr style="height: 56px;">
  <td id="projectlogo"><img alt="Logo" src="../../opencv-logo-small.png"/></td>
  <td style="padding-left: 0.5em;">
   <div id="projectname">OpenCV
   &#160;<span id="projectnumber">4.5.2</span>
   </div>
   <div id="projectbrief">Open Source Computer Vision</div>
  </td>
 </tr>
 </tbody>
</table>
</div>
<!-- end header part -->
<!-- Generated by Doxygen 1.8.13 -->
<script type="text/javascript">
var searchBox = new SearchBox("searchBox", "../../search",false,'Search');
</script>
<script type="text/javascript" src="../../menudata.js"></script>
<script type="text/javascript" src="../../menu.js"></script>
<script type="text/javascript">
$(function() {
  initMenu('../../',true,false,'search.php','Search');
  $(document).ready(function() { init_search(); });
});
</script>
<div id="main-nav"></div>
<!-- window showing the filter options -->
<div id="MSearchSelectWindow"
     onmouseover="return searchBox.OnSearchSelectShow()"
     onmouseout="return searchBox.OnSearchSelectHide()"
     onkeydown="return searchBox.OnSearchSelectKey(event)">
</div>

<!-- iframe showing the search results (closed by default) -->
<div id="MSearchResultsWindow">
<iframe src="javascript:void(0)" frameborder="0" 
        name="MSearchResults" id="MSearchResults">
</iframe>
</div>

<div id="nav-path" class="navpath">
  <ul>
<li class="navelem"><a class="el" href="../../d6/d00/tutorial_py_root.html">OpenCV-Python Tutorials</a></li><li class="navelem"><a class="el" href="../../d2/d96/tutorial_py_table_of_contents_imgproc.html">Image Processing in OpenCV</a></li>  </ul>
</div>
</div><!-- top -->
<div class="header">
  <div class="headertitle">
<div class="title">Image Segmentation with Watershed Algorithm </div>  </div>
</div><!--header-->
<div class="contents">
<div class="textblock"><h2>Goal </h2>
<p>In this chapter,</p><ul>
<li>We will learn to use marker-based image segmentation using watershed algorithm</li>
<li>We will see: <b><a class="el" href="../../d3/d47/group__imgproc__segmentation.html#ga3267243e4d3f95165d55a618c65ac6e1" title="Performs a marker-based image segmentation using the watershed algorithm. ">cv.watershed()</a></b></li>
</ul>
<h2>Theory </h2>
<p>Any grayscale image can be viewed as a topographic surface where high intensity denotes peaks and hills while low intensity denotes valleys. You start filling every isolated valleys (local minima) with different colored water (labels). As the water rises, depending on the peaks (gradients) nearby, water from different valleys, obviously with different colors will start to merge. To avoid that, you build barriers in the locations where water merges. You continue the work of filling water and building barriers until all the peaks are under water. Then the barriers you created gives you the segmentation result. This is the "philosophy" behind the watershed. You can visit the <a href="http://cmm.ensmp.fr/~beucher/wtshed.html">CMM webpage on watershed</a> to understand it with the help of some animations.</p>
<p>But this approach gives you oversegmented result due to noise or any other irregularities in the image. So OpenCV implemented a marker-based watershed algorithm where you specify which are all valley points are to be merged and which are not. It is an interactive image segmentation. What we do is to give different labels for our object we know. Label the region which we are sure of being the foreground or object with one color (or intensity), label the region which we are sure of being background or non-object with another color and finally the region which we are not sure of anything, label it with 0. That is our marker. Then apply watershed algorithm. Then our marker will be updated with the labels we gave, and the boundaries of objects will have a value of -1.</p>
<h2>Code </h2>
<p>Below we will see an example on how to use the Distance Transform along with watershed to segment mutually touching objects.</p>
<p>Consider the coins image below, the coins are touching each other. Even if you threshold it, it will be touching each other.</p>
<div class="image">
<img src="../../water_coins.jpg" alt="water_coins.jpg"/>
<div class="caption">
image</div></div>
<p> We start with finding an approximate estimate of the coins. For that, we can use the Otsu's binarization. </p><div class="fragment"><div class="line"><span class="keyword">import</span> numpy <span class="keyword">as</span> np</div><div class="line"><span class="keyword">import</span> cv2 <span class="keyword">as</span> cv</div><div class="line"><span class="keyword">from</span> matplotlib <span class="keyword">import</span> pyplot <span class="keyword">as</span> plt</div><div class="line"></div><div class="line">img = <a class="code" href="../../d4/da8/group__imgcodecs.html#ga288b8b3da0892bd651fce07b3bbd3a56">cv.imread</a>(<span class="stringliteral">&#39;coins.png&#39;</span>)</div><div class="line">gray = <a class="code" href="../../d8/d01/group__imgproc__color__conversions.html#ga397ae87e1288a81d2363b61574eb8cab">cv.cvtColor</a>(img,cv.COLOR_BGR2GRAY)</div><div class="line">ret, thresh = <a class="code" href="../../d7/d1b/group__imgproc__misc.html#gae8a4a146d1ca78c626a53577199e9c57">cv.threshold</a>(gray,0,255,cv.THRESH_BINARY_INV+cv.THRESH_OTSU)</div></div><!-- fragment --><p> Result:</p>
<div class="image">
<img src="../../water_thresh.jpg" alt="water_thresh.jpg"/>
<div class="caption">
image</div></div>
<p> Now we need to remove any small white noises in the image. For that we can use morphological opening. To remove any small holes in the object, we can use morphological closing. So, now we know for sure that region near to center of objects are foreground and region much away from the object are background. Only region we are not sure is the boundary region of coins.</p>
<p>So we need to extract the area which we are sure they are coins. Erosion removes the boundary pixels. So whatever remaining, we can be sure it is coin. That would work if objects were not touching each other. But since they are touching each other, another good option would be to find the distance transform and apply a proper threshold. Next we need to find the area which we are sure they are not coins. For that, we dilate the result. Dilation increases object boundary to background. This way, we can make sure whatever region in background in result is really a background, since boundary region is removed. See the image below.</p>
<div class="image">
<img src="../../water_fgbg.jpg" alt="water_fgbg.jpg"/>
<div class="caption">
image</div></div>
<p> The remaining regions are those which we don't have any idea, whether it is coins or background. Watershed algorithm should find it. These areas are normally around the boundaries of coins where foreground and background meet (Or even two different coins meet). We call it border. It can be obtained from subtracting sure_fg area from sure_bg area. </p><div class="fragment"><div class="line"><span class="comment"># noise removal</span></div><div class="line">kernel = np.ones((3,3),np.uint8)</div><div class="line">opening = <a class="code" href="../../d4/d86/group__imgproc__filter.html#ga67493776e3ad1a3df63883829375201f">cv.morphologyEx</a>(thresh,cv.MORPH_OPEN,kernel, iterations = 2)</div><div class="line"></div><div class="line"><span class="comment"># sure background area</span></div><div class="line">sure_bg = <a class="code" href="../../d4/d86/group__imgproc__filter.html#ga4ff0f3318642c4f469d0e11f242f3b6c">cv.dilate</a>(opening,kernel,iterations=3)</div><div class="line"></div><div class="line"><span class="comment"># Finding sure foreground area</span></div><div class="line">dist_transform = <a class="code" href="../../d7/d1b/group__imgproc__misc.html#ga25c259e7e2fa2ac70de4606ea800f12f">cv.distanceTransform</a>(opening,cv.DIST_L2,5)</div><div class="line">ret, sure_fg = <a class="code" href="../../d7/d1b/group__imgproc__misc.html#gae8a4a146d1ca78c626a53577199e9c57">cv.threshold</a>(dist_transform,0.7*dist_transform.max(),255,0)</div><div class="line"></div><div class="line"><span class="comment"># Finding unknown region</span></div><div class="line">sure_fg = np.uint8(sure_fg)</div><div class="line">unknown = <a class="code" href="../../d2/de8/group__core__array.html#gaa0f00d98b4b5edeaeb7b8333b2de353b">cv.subtract</a>(sure_bg,sure_fg)</div></div><!-- fragment --><p> See the result. In the thresholded image, we get some regions of coins which we are sure of coins and they are detached now. (In some cases, you may be interested in only foreground segmentation, not in separating the mutually touching objects. In that case, you need not use distance transform, just erosion is sufficient. Erosion is just another method to extract sure foreground area, that's all.)</p>
<div class="image">
<img src="../../water_dt.jpg" alt="water_dt.jpg"/>
<div class="caption">
image</div></div>
<p> Now we know for sure which are region of coins, which are background and all. So we create marker (it is an array of same size as that of original image, but with int32 datatype) and label the regions inside it. The regions we know for sure (whether foreground or background) are labelled with any positive integers, but different integers, and the area we don't know for sure are just left as zero. For this we use <b><a class="el" href="../../d3/dc0/group__imgproc__shape.html#gaedef8c7340499ca391d459122e51bef5" title="computes the connected components labeled image of boolean image ">cv.connectedComponents()</a></b>. It labels background of the image with 0, then other objects are labelled with integers starting from 1.</p>
<p>But we know that if background is marked with 0, watershed will consider it as unknown area. So we want to mark it with different integer. Instead, we will mark unknown region, defined by unknown, with 0. </p><div class="fragment"><div class="line"><span class="comment"># Marker labelling</span></div><div class="line">ret, markers = <a class="code" href="../../d3/dc0/group__imgproc__shape.html#gac2718a64ade63475425558aa669a943a">cv.connectedComponents</a>(sure_fg)</div><div class="line"></div><div class="line"><span class="comment"># Add one to all labels so that sure background is not 0, but 1</span></div><div class="line">markers = markers+1</div><div class="line"></div><div class="line"><span class="comment"># Now, mark the region of unknown with zero</span></div><div class="line">markers[unknown==255] = 0</div></div><!-- fragment --><p> See the result shown in JET colormap. The dark blue region shows unknown region. Sure coins are colored with different values. Remaining area which are sure background are shown in lighter blue compared to unknown region.</p>
<div class="image">
<img src="../../water_marker.jpg" alt="water_marker.jpg"/>
<div class="caption">
image</div></div>
<p> Now our marker is ready. It is time for final step, apply watershed. Then marker image will be modified. The boundary region will be marked with -1. </p><div class="fragment"><div class="line">markers = <a class="code" href="../../d3/d47/group__imgproc__segmentation.html#ga3267243e4d3f95165d55a618c65ac6e1">cv.watershed</a>(img,markers)</div><div class="line">img[markers == -1] = [255,0,0]</div></div><!-- fragment --><p> See the result below. For some coins, the region where they touch are segmented properly and for some, they are not.</p>
<div class="image">
<img src="../../water_result.jpg" alt="water_result.jpg"/>
<div class="caption">
image</div></div>
 <h2>Additional Resources </h2>
<ol type="1">
<li>CMM page on <a href="http://cmm.ensmp.fr/~beucher/wtshed.html">Watershed Transformation</a></li>
</ol>
<h2>Exercises </h2>
<ol type="1">
<li>OpenCV samples has an interactive sample on watershed segmentation, watershed.py. Run it, Enjoy it, then learn it. </li>
</ol>
</div></div><!-- contents -->
<!-- HTML footer for doxygen 1.8.6-->
<!-- start footer part -->
<hr class="footer"/><address class="footer"><small>
Generated on Fri Apr 2 2021 11:36:37 for OpenCV by &#160;<a href="http://www.doxygen.org/index.html">
<img class="footer" src="../../doxygen.png" alt="doxygen"/>
</a> 1.8.13
</small></address>
<script type="text/javascript">
//<![CDATA[
addTutorialsButtons();
//]]>
</script>
</body>
</html>
